Lets Encrypt Apache Tomcat’ 


* Tomcat will not actually be encrypted. 


OR for Slides: 


Christopher Schultz 
CHADIS, Inc. 


ASF Member, Tomcat PMC, Security Team 


https://people.apache.org/~schultz/COCNA2023/Let's Encrypt Apache Tomcat.pdf 


Apache Tomcat 


Java Web Application Server 

Implements J2EE API Specifications “an? Jakarta) 
- Java Servlet 

- Java ServerPages (JSP) 


- Java Expression Language (EL) 
- Java WebSocket 


Apache Tomcat 


e Provides Services 
- Java Naming and Directory Interface (JNDI) 
- JDBC DataSource, Mail Session via JNDI 


e Provides Client Connectivity 
- HTTP, AJP 
- HTTPS using SSL/TLS 


Why Encrypt 


e Security for services that need security 
- Obvious 
e Security for users of sites that do not need security 
- Not so obvious 
- MitM is easy 
- MitM = pwned 
— https:/Awww.troyhunt.com/heres-why-your-static-website-needs-https/ 


Transport Layer Security (TLS) 


e Formerly known as “Secure Sockets Layer” 


e Provides authenticated and confidential conversations 
- Client and server can authenticate each other 
- Conversation is encrypted 


Transport Layer Security 


Client and server negotiate a “cipher suite” 

- Protocol (e.g. TLSv1, TLSv1.2, TLSv1.3, etc.) 

- Authentication (e.g. X.509 with RSA/DSA or EC) 

- Key exchange (e.g. RSA, DHE, ECDHE, etc.) 

- Bulk encryption algorithm (e.g. AES, 3DES, CHACHAZ20, etc.) 
- Message authentication code (e.g. SHA-1, SHA-2, etc.) 


Public Key Infrastructure 


° — Trust Model 
Server produces certificate 
- Server authenticates to Certificate Authority (CA) 
- Certificate Authority signs Server’s certificate 


- Server presents CA-signed certificate to client when a client initiates 
a connection 


- Client trusts the Certificate Authority 
- Client therefore trusts Server 


Public Key Infrastructure 


mee Trusts > Web Server 
Signs = Server Cert 


(Inherits 
Trust) 


Public Key Infrastructure 


Certificate Authorities 
- Have nearly universal (client) trust 


- Provide multiple levels of authentication 
e Domain-Validated (DV) 
e Organization-Validated (OV) 
e Extended Validation (EV) 


— Require human interaction for requests, issuance 
- Issue certificates for several years 
- Charge a fee for a issuance 


Let's Encrypt 


e Wanted widespread TLS 
- Free 
- Easy 
- Makes the Web a safer place 
e Questioned CA’s 
- Signing-request and issuance processes 
- Fees for freely-available crypto 


e Built a better mousetrap 


Let's Encrypt 


Near-universal trust 
- Cross-signed certificate from IdenTrust (an existing CA) 
- Most browsers and OSs now include LE root certs 


Provides a single level of authentication 
- Domain-Validated 


Requires automated interaction for requests, issuance 
Issues certificates valid for 90-day intervals 
Charges no fee for issuance 


Let's Encrypt 


e Not replacing CAs 
- No Organization-Validation or Extended-Validation certificates 
- No code- or email-signing certificates 


e Merely reduces the financial barrier for mundane TLS to zero 


The Plan 


e Once 

- Request a certificate from Let’s Encrypt 
e Periodically (~50 day intervals) 

- Request a certificate renewal 

- Deploy the new certificate into Tomcat 


The Plan 


e Request a certificate from Let’s Encrypt 

- Easy: use EFF’s certbot tool (or your ACME client of choice) 
e Periodically request a renewal 

- Easy: Use cron + EFF’s certbot tool 
e Install the new certificate into Tomcat 

- Not straightforward 


Tomcat Troubles 


e Tomcat usually doesn’t bind to port 80 
— Might be tricky to renew certificates 

e Tomcat uses Keystores’ 
- certbot produces plain-old PEM files 

e Tomcat’s “graceful reload” isn’t Super convenient 
— httpd has this, and certbot uses it 


“PEM files can also be used. 


Tomcat Solutions 


e Port binding 
=- jsve 
- iptables 
e Java Keystores 
- Can import PEM files 
e Tomcat reloads 
- Can be done 
- Without downtime 
- In-process requests will complete \o/ 


Getting that first LE Cert 


e iptables 
- More than just a firewall 
- Can perform routing and forwarding 
- Need a few commands to redirect port 80 — 8080 


Getting that first LE Cert 


° — magic sauce 
NAT PREROUTING 80 — 8080 
- NAT OUTPUT 8080 — 80 
- NAT PREROUTING 443 — 8443 
- NAT OUTPUT 8443 | 443 
- Also may require: 
- FILTER FORWARD 80 ACCEPT 
- FILTER FORWARD 443 ACCEPT 


Getting that first LE Cert 


e iptables magic sauce 


- HTTP 


° iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 
8080 


e iptables -t nat -A OUTPUT -o lo -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 
8080 
— HTTPS 


e iptables -t nat -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 
8443 


e iptables -t nat -A OUTPUT -o lo -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 
8443 


Getting that first LE Cert 


e iptables magic sauce 


- Also might need 
e iptables -A FORWARD -p tcp -m tcp —dport 80 -j ACCEPT 
e iptables -A FORWARD -p tcp -m tcp —dport 443 -j ACCEPT 


Getting that first LE Cert 


e Now we can run certbot-auto to get a new certificate 


certbot-auto certonly --webroot \ 

--webroot-path “S{CATALINA_ BASE Vwebapps/ROOT” \ 
-d www.example.com \ 

--rsa-key-size 4096 


Reconfiguring Tomcat’s TLS 


e Start with self-signed certificates 
- keytool -genkeypair \ 
-keystore conf/keystore.pl12.1 \ 
“storetype PRKCSLZ: 3 
-alias tomcat -keyalg RSA \ 
-sigalg SHA256withRSA \ 
-keysize 4096 -validity 10 


- Hostname: localhost 
- Organizational Unit: Keystore #1 


Reconfiguring Tomcat’s TLS 


e Generate a second keystore 
- keytool -genkeypair \ 
-keystore conf/keystore.pl2.2 \ 
“storetype PRKCSLZ: 3 
-alias tomcat -keyalg RSA \ 
-sigalg SHA256withRSA \ 
-keysize 4096 -validity 10 


- Hostname: localhost 
- Organizational Unit: Keystore #2 


Reconfiguring Tomcat’s TLS 


Symlink conf/keystore.p12.1 — conf/keystore.p12 
Configure the connector in Tomcat 


- <Connector port="8443” keystoreFile="conf/keystore.p12” ... 


Start Tomcat 


Verify connection 
- openssl s_ client -no_ssl3 -connect localhost:8443 
- openssl s_ client -no_ssl3 -connect localhost:443 


[> 


Reconfiguring Tomcat’s TLS 


e Remove existing symlink 
e Symlink conf/keystore.p12.2 — conf/keystore.p12 
e Now what? 


Reconfiguring Tomcat’s TLS 


Tomcat 

- Exposes ProtocolHandlers via JMX 
ProtocolHandlers via JMX 
reloadSslHostConfigs 

- ... In Tomcat 8.5.32+ 

- ... or Tomcat 9.0.3+ (?) 

- ... and later versions 


MBeans 


v B Catalina 
> P Connec tor 
> P DataSource 
> P Deployer 


> Sf NamingResources 
« P ParalleWebappClassLoader 
v E ProtocolHandler 
Ñ 8215 
v B 8217 
S "127.0.0.1" 


Reconfiguring Tomcat’s TLS 


Connect to Tomcat via JMX 
Navigate to the proper —— 


MBeans Attributes |; Operations | Notifications | Metadata 
P rotoco | | | an d le r v B Catalina Operation invocation 
> P Connector : 
void 
> P DataSource start O 
> P Deployer 


v reloadSslHostConfigs O 


> Sl NamingResources 


Invoke the 
reloadHostConfigs operation + Eremi e =~ 


Ñ 8215 
v B 8217 void pause () 


Verify Connection 5°127.0.0.1" 
- openssl s_ client -no_ssl3 -connect localhost:443 


Reconfiguring Tomcat’s TLS 


e Manual Deployment 
- Inconvenient (VisualVM in production?) 
- Time-consuming 
- Required with irritating frequency 
e every few weeks 
e for every server 


- Doesn't scale 


Reconfiguring Tomcat’s TLS 


e Automation is Required 
1. Renew certificate from Let’s Encrypt (certbot) 
2. Build a new keystore (openssl) 
3. Reload Tomcat’s Keystore 


Let’s Encrypt Renewals 


e Invoke certbot-auto renew 
e Celebrate! 


Build a new Keystore 


e Package server key and certificate into PKCS#12 file 


- openssl pkcsi2 -export -in [cert] -inkey [key] -certfile [chain] -out 
[pL 2file] 


e Celebrate! 


Reload Tomcat’s Keystore 


e Tomcat Manager to the Rescue 
- JMXProxyServlet 
e Enable Manager Application 


- Need to configure a <Realm> 
e Security! 


Reload Tomcat’s Keystore 


e Invoke reload method 


- curl https://localhost/manager/jmxproxy?invoke=Catalina%3Atype 
%3DProtocolHandler%2Cport%3D8443%2Caddress%3D 
%22127.0.0.1%22&0p=reloadSslHostConfigs 


e Celebrate 


Automated Deployment 


e Scripting’ will set you free 
- certbot-auto renew 


- openssl pkcs12 -export -in [cert] -inkey [key] -certfile [chain] -out 
[p12file] 

- curl https://localhost/manager/jmxproxy?invoke=Catalina%3Atype 
%3DProtocolHandler%2Cport%3D8443%2Caddress%3D 
%22127.0.0.1%22&0p=reloadSslHostConfigs 


* The actual script has a lot more detail that won't fit here. 


Bonuses 


e Allows CRL reloading (if you like that kind of thing) 
¢ Allows on-the-fly TLS reconfiguration 
- Protocols 
- Cipher suites 
¢ Allows additional certificates to be added (e.g. EC) 
- ... anything else encapsulated by the SSL engine 


Bonuses 


e Will work for all connector types 
- NIO/NIO2 
- APR 


Let’s Encrypt Apache Tomcat 


Let’s Encrypt provides free (beer) certificates 
Automation is required for issuance and renewal 
Tomcat is somewhat more complicated than e.g. httpd 
Those complications can be overcome 


Questions 


https://people.apache.org/~schultz/presentations/COCNA2023/Let's Encrypt Apache Tomcat.pdf 
Sample code available in the same directory. 


